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Drawing from a design and development research approach, specifically model research, this study 
investigated the perspectives of higher education faculty and administrators regarding their 
experiences with a university-wide electronic portfolio implementation initiative. Participants in the 
study were fifty-two faculty and administrators at a large research university in the United States 
who were either continued users or recent abandoners of electronic portfolios. Survey and interview 
data were used to understand participant perspectives on the electronic portfolio implementation 
process, including perceived enablers and barriers to adoption of this instructional technology. Study 
findings and Diffusion of Innovation (DOI) theory informed the development of a six-component 
electronic portfolio implementation framework. Three external experts in systemic change were then 
asked to review the framework. Feedback from these external experts was incorporated into a 
revised version of the framework that is presented here. The framework can be used by an 
educational institution to support the successful adoption and integration of electronic portfolios 
regardless of where the organization is within the implementation process. 


In recent years, higher education has witnessed an 
increase in the availability and use of electronic 
portfolios (ePortfolios) to support learning, assessment, 
and professional development. ePortfolios offer a 
unique way to capture a variety of learning evidence 
from students over time in multiple formats and across 
varied contexts, while also gaining students’ personal 
reflections on individual learning and growth (Chen & 
Light, 2010). ePortfolios enable instructor insight into 
student mastery of knowledge as well as fluency with 
technology (Chen & Light, 2010). As an instructional 
technology, ePortfolios offer customized approaches to 
learning and assessment through the integration of 
varied technologies that provide more choices for 
students and educators; thus, broadening opportunities 
for pedagogical change in higher education contexts. 

While essential to successful ePortfolio adoption 
and implementation, higher education faculty are 
engaged rarely as active participants in an open 
innovation process (C. E. Watson, personal 
communication, January 19, 2012). Further, whereas 
student perspectives of ePortfolio adoption are well 
represented in the literature, faculty perspectives are not 
(Ruiz, Quadri, & Karides, 2009; Wang & Turner, 
2007). Yet it is well documented that the involvement 
of all key stakeholders throughout an entire change 
process is important to its success (Patton, 2014; Russ- 
Eft & Preskill, 2009). The purpose of this study was to 
investigate faculty and administrators perspectives 
regarding the university-wide implementation of an 
ePortfolio initiative in order to develop an 
implementation framework that integrates their voices 
in light of systemic change theory. Two research 
questions supported this work: 


• How do faculty and administrators perceive 
the ePortfolio adoption process? What about 
the process is successful? What about the 
process is lacking and requires improvement? 
What about the process reflects Diffusion of 
Innovation (DOI) theory? 

• In light of participant experiences, what 
features of DOI theory should be included in 
an ePortfolio implementation framework? 

The resulting ePortfolio implementation 
framework, as a support for instructional technology 
innovation management across an organization, informs 
higher education policy, administration, and process. In 
addition, the framework transforms ePortfolio 
implementation into a more accessible and feasible 
endeavor for faculty interested in ePortfolio adoption 
and use but at a loss for how to enact, as well as sustain, 
this innovation. 

Conceptual Framework 

This study was informed by two major conceptual 
areas: the evolution and use of portfolios, and 
specifically ePortfolios, to support learning, assessment, 
and professional development and the adoption and 
implementation of an innovation such as ePortfolios 
through the lens of DOI theory. 

The Evolution and Use of Portfolios 

The use of portfolios to demonstrate mastery of 
knowledge and skills is not new to education. Writers, 
artists, builders, and more have used portfolios to 
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collect, document, and share growing bodies of work as 
they developed in knowledge and skill. To be sure, 
portfolio use spans content areas and dates back 
centuries (Adams, 2010). Formal portfolio work can be 
dated back to Leonardo Di Vinci, who diligently kept a 
portfolio to document his inventions, thoughts, and 
reflections (H. Barrett, personal communication, 
August 14, 2014). As portfolios have shifted to 
electronic formats, a renewed interest in their adoption 
and implementation has led to new opportunities for 
learning, assessment, and professional development. 

Assessment of student learning will continue to 
gain importance in ongoing educational reform 
efforts (Baker, 2001). Assessment approaches must 
continue to advance if they are to inform individual 
student learning in more dynamic and sophisticated 
ways. While traditional assessments such as exams 
are often considered efficient to administer and 
grade, these types of assessments typically focus on 
the acquisition of foundational knowledge and are 
unable to assess higher-level knowledge and skills 
(Linn, 1993). However, performance assessments, 
such as those included in a portfolio of work, 
require observable disciplinary activity and artifacts 
(Davies & Le Mahieu, 2003; Linn, 1993) and 
empower a learner to exhibit the development of 
new knowledge and skills over time, offering 
greater depth and complexity (Airasian, 1996). 

Watson, Zaldivar, and Summers (2010) claim 
that ePortfolios assist with assessment of students 
on three distinct levels. First, the creation process 
for building ePortfolios provides a method for 
capturing student learning that is often unable to be 
captured using traditional assessment, allowing 
instructors to see the growth of students through a 
course or program. Second, if the instructor of a 
course or program builds their own ePortfolio 
alongside students, that instructor will be able to 
better reflect on the progress and experiences of 
their students. Lastly, programs and institutions 
also benefit from the use of ePortfolios, providing 
rich learning and program assessment data that can 
be used for curricular improvements. 

Over the last decade, the versatility, portability, 
and efficiency of ePortfolios have brought prominence 
to this instructional technology in higher education 
across disciplines. While this trend may originate in 
the need to assess students and student work in diverse 
ways, ePortfolios have also proven useful for 
examining and supporting individual learning and 
professional development over time (Mitchelson & 
Mandell, 2004; Watson & Doolittle, 2011). In recent 
years, the use of ePortfolios has continued to increase 
at the undergraduate level in higher education 
(Dahlstrom & Bichsel, 2014). 


ePortfolio Adoption and Diffusion of Innovation 
Theory 

As higher education institutions increase their use 
of instructional technologies, ePortfolios meet a 
growing institutional need for relevancy to the teaching 
and learning enterprise (Bass & Eynon, 2009; 
Schneider, 2009). However, similar to any other 
innovation, ePortfolios are subject to the conditions and 
stages of the innovation diffusion process and barriers 
to their adoption, integration, and sustainment arise 
(Annan, 2008; Surry, 2002). In addition, the meaningful 
and purposeful implementation of ePortfolios on a large 
scale can be challenging (Cambridge, 2012). Applying 
what is known about technology adoption and diffusion 
to the introduction of ePortfolios into an institutional 
system can support integration while still honoring the 
unique perspectives and contexts of local faculty users. 

Diffusion of innovation theory (DOI), which seeks 
to understand the social process that community 
members engage in to adopt or reject an innovation 
(Rogers, 2003; Surry & Farquhar, 1997), was relied on 
as the broad conceptual framework for guiding all 
aspects of the study’s design including instrument 
development, data collection and analysis, framework 
development, and framework review and revision. 
Specifically, this study drew from two DOI theoretical 
perspectives: Rogers’ five Stages of Adoption and Ely’s 
Eight Conditions for Change. 

Rogers (2003) identified five Stages of Adoption of 
an innovation: knowledge, persuasion, decision, 
implementation, and confirmation. Similarly, Ely 
(1990) described Eight Conditions for Change: 
dissatisfaction with the status quo, sufficient knowledge 
and skills, availability of resources, availability of time, 
rewards or incentives, participation, commitment, and 
leadership. Drawing from Rogers and Ely, Surry and 
Farquhar (1997) argued that the study of an 
instructional technology in light of DOI theory can help 
instructional designers have a better understanding of 
the adoption or rejection of an innovation, work more 
effectively with clients, and even “lead to the 
development of a systematic model of adoption and 
diffusion” (p. 2). Grounded in DOI theory and his own 
professional experiences with innovation adoption, 
Surry developed the RIPPLES survey as a means for 
studying the adoption of an instructional technology 
across seven dimensions of DOI: resources, 
infrastructure, people, policies, learning, evaluation, 
and support. This study employed a modified RIPPLES 
survey (Blevins & Brill, 2013), along with selected 
follow-up interviews, to explore the perspectives of 
faculty and administrators experienced with the 
adoption of ePortfolios at a large university to inform 
the development of an implementation framework. 
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Methodology 

Study Design 

This study drew from a design and development 
research methodology, defined by Richey and Klein 
(2007) as the “systematic study of design, development 
and evaluation processes with the aim of establishing an 
empirical basis for the creation of instructional and non- 
instructional products and tools and new or enhanced 
models that govern their development” (p. 1). More 
specifically, this effort used what was previously known as 
Type 2 developmental research, recently renamed to 
model research, in which the research “pertains to the 
[study] of the development, validation, and use of design 
and development models.” (Richey & Klein, 2007, p. 10). 

Consistent with model development research, three 
phases informed framework development: analysis, 
development and evaluation, and revision. In the 
analysis phase, faculty and administrator perspectives 
about the ePortfolio implementation process were 
investigated through survey and interviews shaped by 
DOI theory. In the development and evaluation phase, 
study findings and DOI theory guided the development 
of a six-component framework that was then evaluated 
by three experts in systemic change. In the revision 
phase, recommendations from the experts directed the 
revision and finalization of the framework. An 
overview on how these phases were applied in this 
study is provided in Table 1. 

Setting and Participants 

A large United States research university with 
approximately 30,000 students began a university-wide 
initiative to implement ePortfolios in 2002. During this 
time, several credible strategies were considered to 
support the initiative’s success including: strategic 
alignment of the initiative to department, college, and 
institutional goals; partnerships with key stakeholders; 
pilot-testing; faculty development opportunities; and 
the use of the Concerns-Based Adoption Model 
(CBAM) for change (Hord, Rutherford, Huling-Austin, 
& Hall, 2006) and Ely’s Eight Conditions of Change 
(Ely, 1990) for implementation (Watson et al., 2010). 

Anecdotal data suggested challenges with long-term 
ePortfolio implementation. However, empirical data to 
investigate the success of these strategies and potentially 
guide improvements did not exist. Thus, approximately 10 
years after the initial implementation of ePortfolios on the 
university’s campus, the perspectives of 144 members of the 
university community were sought through survey and 
follow-up interviews. These members of the university 
community were identified through the university’s 
ePortfolio office as faculty and administrators who had used 
or were currently using ePortfolios. 


Data Sources and Analysis 

The survey instrument was a modified RIPPLES 
survey, which is based in part, in DOI theory and 
specifically designed to explore instructional technology 
integration in higher education. The 55 question survey was 
divided into four sections: participant demographics; 
background (individual historical use of ePortfolios); 
ePortfolio implementation at their university; and, opinion 
of ePortfolio adoption and implementation. The ePortfolios 
at the university section, which was modified to more 
directly reflect Ely’s Eight Conditions for Change, 
contained seven subsections reflective of the RIPPLES 
model acronym: resources (time and money); infrastructure; 
people (communication and shared decision-making); 
policies; learning (specific instructional outcomes for user 
training); evaluation; and (user) support. Each of the seven 
subsections contained close-ended questions as well as at 
least one open-ended question. The close-ended question in 
these sections had a possible value between one and six (1 = 
don’t know/unsure; 2 = strongly disagree; 3 = disagree; 4 = 
neutral; 5 = agree; 6 = strongly agree). 

Fifty-two out of 144 individuals (36%) responded 
to the survey. Typical response rates for online surveys 
are 52%, plus or minus 20% (Baruch & Holtom, 2008). 
Thus, while the response rate was lower than desired, it 
can still be considered acceptable. A descriptive 
analysis of the data was conducted first in order to 
determine the means, percentages, and standard 
deviations for each survey item. Second, participants’ 
answers to the open-ended questions were examined for 
emerging themes (Creswell, 2009). The survey findings 
influenced the development of the final interview 
protocol in order to provide opportunities for more 
directed data collection based upon the study’s purpose. 

The ten-question interview protocol probed each 
participant to speak in greater depth about their 
experiences implementing ePortfolios at the university. 
A small interview sample of 12 survey participants was 
selected to represent a diverse cross-section of the 
university. Selection criteria included: discipline, gender, 
years at the university, years teaching, role (faculty or 
administrator), time using ePortfolios, and current usage 
status (continued user or abandoner). Interview 
transcripts were coded for themes. Interview findings 
were then triangulated with participant survey findings in 
order to strengthen the analytic process (Creswell, 2009). 

Using the findings from the survey, interviews, and 
DOI literature review, a framework for supporting the 
adoption of ePortfolios by university faculty, staff, and 
administrators was developed. Conceptually, the 
framework was meant to operationalize those aspects of 
DOI theory that appeared to be most supportive of 
successful ePortfolio adoption. Five experts in DOI 
theory were asked to provide feedback, via a rubric, 
regarding the extent to which the framework effectively 
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Table 1 

Overview of Study Phases 

Phases 

Framework Development and Validation 

Analysis 

Analyze DOI Literature and create survey and interview protocol to collect 
participant data. 

Analyze survey data, interview data, and DOI literature. 

Development and Evaluation 

Develop framework based on analysis. 

Develop rubric for DOI expert reviewers. 

Administer expert reviews. 

Revision 

Analyze expert reviews. 

Incorporate feedback from expert reviewers to create a revised framework. 


and appropriately integrated important DOI elements. 
Three reviewers completed the review process. This 
feedback was analyzed and incorporated into a revised 
ePortfolio implementation framework. 

Findings 

Participant Demographics 

Fifty-two out of 144 individuals responded to the 
survey (36%), and all of them indicated that they were 
currently or had previously used Sakai, the university’s 
ePortfolio system. Sixty-two percent (32) of the 
participants who submitted the survey were female, and 
38% (20) were male. In response to age, 4% (2) 
indicated they were age 20-29; 10% (5) selected age 
30-39; 27% (14) indicated age 40-49; 38% (20) 
identified as 50-59; 17% (9) selected age 60-69; and 
4% (2); age 70 or above. Thus, based on age alone, 
14% of respondents could be considered early career, 
while 69% could be characterized as mid to late career. 

Regarding professional position, 67% (34) of 
survey respondents were faculty with at least some 
teaching responsibilities and 29% (15) were in 
administrative roles. When asked the number of years 
teaching at the college or university level, 35% (18) 
answered zero to 10 years; 40% (21) answered 11 to 25 
years; and 25% (13) answered 25 years or more. 

ePortfolio Use 

When asked how long participants had been using 
or previously used ePortfolios, 48 of 52 participants 
(92%) responded. Of those responses, 23% (11) 
answered less than one year; 35% (17) answered one to 
three years; and 42% (20) answered four or more years. 
Surprisingly, 42% of respondents reported abandoning 
the use of ePortfolios. When asked why they stopped 
using them, 22 of 52 participants (42%) responded. 
Responses were grouped into the following six 


categories, ordered here from high to low: change in 
employment position (8); usability and reliability of 
technology (8); faculty or student resistance (3); too 
much time or effort required (3); change in course 
structure (3); and, still in development (1). Of note here 
are the two categories of technology usability and time 
investment in that these themes also arose in other 
sections of the data. 

Regarding the purpose(s) for using ePortfolios, 50 
of 52 participants (96%) responded. Of those responses, 
58% (29) answered to track learning; 60% (30) 
answered to assess learning; 40% (20) answered to 
support professional development; and 36% (18) 
answered Other. From the Other category, the 
following response themes emerged: scholarship and 
employment (4); course or program requirement (3); 
showcase student work (3); and accreditation (1). Thus, 
most respondents rely on assessment and the tracking of 
learning as the main reasons for using ePortfolios. 

When prompted to identify what they liked most 
about using ePortfolios, 50 of 52 participants (98%) 
responded. Ordered high to low, these categories 
included: housing and showcasing of artifacts (27); self- 
reflection and learning process engagement (15); meets 
accreditation and assessment requirements (7); reveals 
whole picture of student (6); flexibility (4); and 
availability and security (1). Thus, most survey 
respondents value ePortfolios as a means to store and 
access student work. A comment by Professor Adams 
(Instructor) reflected how ePortfolios have been of value: 

Prior to [ePortfolios], we were doing [artifact 
creation and collection] in different areas. We had 
a piece here, a piece here, and we were trying to 
teach the [students] a methodology of developing 
themselves, but in addition to that, ‘How can I 
prepare myself for finding a job?’ 

When asked to pinpoint what they liked least about 
using ePortfolios, 51 of 52 participants (98%) 
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responded. Responses were grouped in categories, high 
to low, as: lack of user-friendly electronic platform 
(33); time spent planning and grading (11); student and 
faculty difficulty and resistance (9); defining and 
understanding ePortfolios (2); and inaccessibility after 
graduation (2). Given these responses, survey 
respondents appear most troubled by the limitations of 
the current ePortfolio system, Sakai. 

Regarding what participants perceived as the most 
important factor(s) influencing faculty adoption and use 
of ePortfolios, 49 of 52 participants (94%) responded. 
Responses were categorized as follows: usability and 
flexibility of system (20); faculty buy in to a clear 
purpose (19); support and training (7); reward for use 
and time commitment (6); and, the learning curve (5). 
Consistent with prior responses, quality of the 
ePortfolio system and user buy-in to a clearly 
communicated purpose arose as the top two factors in 
ePortfolio adoption. 

ePortfolios at the University 

To investigate more closely how participants viewed 
the seven DOI factors previously identified in the 
RIPPLES model, they were asked to rate the importance 
of each of these factors in regards to ePortfolio 
implementation at their university as well as ePortfolio, 
as a representative instructional technology, for adoption 
and implementation in general. Regarding the 
importance of each individual RIPPLES item to 
implementation, participants rated Infrastructure of 
greatest importance (94% agree or strongly agree); 
Resources (time and money) of second greatest 
importance (92%); Learning (specific instructional 
outcomes of user training) in third place (87%); and User 
Support as fourth (85%). The rest of the items fell in line 
as follows: Policies (60%), Evaluation (66%), and People 
(communication and shared decision-making) (52%). 

Given that Infrastructure was rated of greatest 
importance, it is not surprising that participants chose to 
comment on infrastructure issues the most on both the 
survey and during the interviews. Survey data revealed 
that, while the overall university infrastructure is 
viewed positively, the ePortfolio technologies are not. 
In fact, the ePortfolio technology system’s design was 
identified by survey participants as the top barrier to 
ePortfolio use. As Professor Johnson (administrator) 
put it during his interview, “You cannot have a 
successful portfolio program if you have a product that 
is full of holes and bugs.” Interestingly, when asked on 
the survey what they felt was the greatest potential 
enabler to ePortfolio use, participants identified the 
technology’s capabilities as second only to support. 

While 92% of survey respondents ranked the 
Resources of time and money as second most important 
to ePortfolio implementation, they distinguished time as 


more important than money to successful use. In fact, 
they identified a lack of time to learn about and 
implement ePortfolios as the second most significant 
barrier to use. In his interview, Professor Lewis 
(Associate Professor) commented, “You have to have 
time to be able to think through the process. There has 
to be time dedicated to the instruction of the technology 
itself and the support of that technology.” Professor 
Young shared, “If you put a lot of technology into your 
class, you get a pat on the head...Nobody’s saying to 
me, ‘Oh here, let me give you fewer classes or 
something to make up for the time you are spending.’” 

In regards to RIPPLES, user Learning (87%) and 
user Support (85%) were rated as third and fourth most 
important respectively to ePortfolio implementation. At 
the university, one central office was dedicated to 
providing both training and support to ePortfolio 
adopters across the campus. This type of centralized 
support was viewed as both essential and exemplary by 
survey respondents. Comments included the following: 

• “The university office responsible for 
administering ePortfolio support is excellent. 
They are always very helpful” (Lecturer). 

• “ The eP office group is great. They have been 
extraordinarily helpful” (Instructional 
Faculty). 

• “The eP office is a lifesaver. Without those 
folks and their support, I would not have 
included ePortfolios” (Instructional Faculty). 

In contrast, elements of support at the program, 
department, and university levels, particularly in 
regards to leadership, were also viewed as essential but 
inconsistent and disjointed. In his interview, Professor 
Adams (Instructor) addressed this need for a more 
unified culture of support: “We all need to be aligned in 
the goals of the ePortfolio...there’s a lot of moving 
parts, and a lot of people need to be on board for it to 
work.” Recall that survey respondents identified faculty 
buy-in to a clear purpose as the second most important 
adoption factor. 

The ePortfolio Implementation Framework 

Based on the findings from survey and interview 
data, as well as a review of the DOI literature, a 
framework for implementing ePortfolios was created. 
This original framework was reviewed by three external 
DOI experts. These experts were selected based on their 
expertise in technology integration and systemic 
change. In addition, all three had experience working 
within higher education as either faculty or 
administrators. Their feedback was then analyzed and 
incorporated into a revised framework. Generally, the 
reviewers agreed that the framework had strong 
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Figure 1 

ePortfolio Implementation Framework components 



Motivation 

Identification and/or 
presence of intrinsic 
and extrinsic incentives 
for electronic portfolio 


Leadership 

The necessary 
leadership support 
in place to sustain 


use of electronic 


oortfolios 


Commitment 

The decision, as a result 
of value recognition, to 
implement electronic 
portfolios 


► 

► 


alignment with DOI theory. They also agreed that the 
framework would prove useful for its intended 
audience. Reflective comments include: “For 
institutions new to ePortfolio, this framework will 
provide much needed guidance and systematic 
recommendations for moving an adoption campaign 
forward” (Reviewer 1); “The framework provides a 
guidance process for implementing and sustaining 
ePortfolio in higher education” (Reviewer 2); “Great 
potential and practical use in the field” (Reviewer 3). 
Reviewers also agreed that the purpose of the 
framework and use of the rating scale could be clarified 
with concerns addressed through the addition and 
revision of some of the framework’s column headings 
and descriptive text. This section describes the revised 
framework in detail. 

The framework developed is meant to support 
those implementing, or attempting to implement, 
ePortfolios in a higher education context by guiding 
them through key attributes of systemic innovation in a 
practical and applied manner. First, six essential 
components were identified and defined through both 


the DOI literature, specifically Rogers (2003) and Ely 
(1990), and study findings. The framework was then 
assembled to include these six components in a modular 
format: awareness, motivation, commitment, resources, 
leadership, and evaluation (see Figure 1). 

Awareness is defined as the professional 
knowledge of the pedagogical benefits of 
ePortfolios. The Awareness component reflects 
Rogers’ (2003) knowledge stage in his Stages of 
Adoption model and Ely’s (1990) dissatisfaction 
with the status quo and sufficient knowledge and 
skills conditions in his Conditions for Change 
model. Study findings demonstrate that participants 
had developed an awareness of the usefulness of 
ePortfolios, especially to capture and show student 
work and as a means for assessment. Further, 
through the centralized ePortfolio office, adopters 
had opportunities to knowledge and skill-build and 
viewed these support experiences as positive. 
Participants also saw the importance of a clear 
purpose for ePortfolio use, an awareness goal that 
can be supported through professional development. 
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The Motivation component of the framework is 
defined as the identification and/or presence of intrinsic 
and extrinsic incentives for using ePortfolios. The 
Motivation component reflects Rogers’ (2003) persuasion 
stage in his Stages of Adoption and Ely’s (1990) 
dissatisfaction with the status quo and need for 
rewards/incentives conditions in his Conditions for 
Change model. A remark by Professor Johnson 
(Administrator) pinpoints a recognized intrinsic value to 
ePortfolios, the move from unwieldy paper-based to more 
manageable electronic means for storage and access: 

Because again, you’ve got a portfolio [this] thick 
for every student in the department and, you know 
we were graduating at that point 20 to 25 students a 
year. Twenty or 25 students a year was three 
quarters of a drawer and after 10 years we had ... a 
lot of records and so ... we were very eager to see 
the ePortfolio and we participated from the very 
beginning. 

Although participants recognized the intrinsic value 
of ePortfolios, they also recognized the significant time 
investment necessary to implement and that such an 
investment should be acknowledged and even mitigated 
or compensated through extrinsic rewards such as a 
course release or graduate assistant support. 

Commitment, the third component in the 
framework, is defined as the decision, as a result of 
value recognition, to implement ePortfolios. The 
commitment component reflects Rogers’ (2003) 
decision stage in his Stages of Adoption and Ely’s 
(1990) participation and commitment conditions in his 
Conditions for Change. The need for consistent 
commitment across program, department, and 
university levels was evident in study findings. As 
Associate Professor Lewis, remarked, “We all need to 
be aligned in the goals of the ePortfolio... there’s a lot 
of moving parts, and a lot of people need to be on board 
for it to work.” 

The next component, Resources, is defined as 
identified resources to assist in ePortfolio 
implementation. This component reflects Rogers’ 
(2003) implementation stage in his Stages of Adoption 
and three conditions in Ely’s (1990) Change model: 
sufficient knowledge and skills, availability of time, 
and availability of resources. Study findings supported 
the importance of adequate resources and resource 
allocation, including adequate time and support, for 
successful ePortfolio implementation. As Professor 
Clark (Administrator) commented: 

“We ran into a whole lot of resource issues, no 
one had the time to work on it even though we 
had leadership buy-in. Resources were not 
provided to back it up even though I think 


[faculty] were interested in it. They felt 
overwhelmed all the time.” 

Leadership is defined as the necessary leadership 
support in place to sustain use of ePortfolios. This 
component reflects Ely’s (1990) leadership condition in 
his Conditions for Change. Study findings supported 
the idea that ongoing involvement and support from 
leadership at all levels is important to sustaining 
ePortfolio implementation. As Professor Johnson 
(Administrator) remarked: 

“You need to make sure that the faculty are aware 
of the opportunity and how easy it to use. I do not 
see much information coming across my desk 
anymore that says, “Hey we have this cool tool, 
why don’t you try it?” 

The final component, Evaluation, is defined as 
the data-based examination of ePortfolio use for 
improvements to future iterations. This framework 
component reflects Roger’s (2003) stages of 
implementation and confirmation. Further, it is also 
reflective of the need for systemic evaluation of the 
ePortfolio initiative, as documented in study 
findings. One survey respondent, an administrator, 
noted the following: 

“I think evaluation is very important. Evaluation 
results need to be communicated and acted upon in 
order for them to be enablers. I think if evaluations 
are done in a solitary way and not acted upon, I am 
not sure how helpful they are.” 

Lumsden (2007) reports evaluation as one of five 
success factors to the university-wide implementation 
of ePortfolios at Florida State University, providing 
further support for including an Evaluation component 
in the framework. 

After the six essential components were identified 
and arranged, more work was done to expand the 
framework into a usable resource (see Appendix) that 
anyone considering implementing ePortfolios, or 
already in the process of implementation, could use to 
assess the workgroup’s current status in the 
implementation process as well as critical next steps. In 
addition to providing a definition of each component, 
guidance in the following areas was provided for each 
component: Selected Strategies to Support Component, 
Key Stakeholder Involvement, Assessment of Current 
Implementation Status, and Next Steps for 
Implementation Efforts. The “Selected Strategies to 
Support Component” column offers selected strategies 
to act on each component. The “Key Stakeholder 
Involvement” column identifies stakeholders that can 
impact progress on that component. A rating scale is 
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provided in the “Assessment of Current Implementation 
Status” column for users to assess where a workgroup 
stands with each component and identify next steps for 
implementation. A rating of one (1 = low) would 
identify a component as a priority in planning efforts, 
whereas a three (3 = high) would indicate the 
component is well-attended to and therefore of low 
priority. Through such a quick check, action planning 
provided in the “Next Steps for Implementation 
Efforts” column could then be based on top priorities, 
perhaps minimizing time required toward adoption and 
implementation efforts. 

Summary 

Study findings resulted in an understanding of 
faculty and administrator perspectives as participants in 
an ongoing university-wide ePortfolio implementation 
that, through the lens of DOI theory, were used to 
construct a framework that can be used by higher 
education community members to enable such a 
systemic initiative. The ePortfolio framework consists 
of six essential elements: awareness, motivation, 
commitment, resources, leadership, and evaluation,. 
Importantly, the framework is modular, not linear, with 
individual elements taking on varied degrees of 
emphasis at different stages in the innovation life cycle. 
An action planning tool accompanies the framework to 
support faculty implementation efforts over time. To 
ensure the framework was appropriately aligned with 
DOI theory in addition to practitioner experiences, it 
underwent expert review by three systemic change 
scholars. Reviewer feedback was then incorporated into 
the final version of the framework in Appendix. 

Discussion 

A university-wide ePortfolio implementation is a 
complex undertaking that requires the long-term and 
attentive coordination of infrastructure, resources, and 
people. This study contributes to the instructional design and 
technology field in two ways. First, it offers a framework for 
ePortfolio adoption and implementation in higher education 
contexts that acknowledges and includes the perspectives of 
faculty and administrators while addressing key elements of 
DOI theory. Second, the study contributes to a small but 
growing collection of design and development research 
studies, providing an example of what this newer 
methodological approach can look like in practice. 

A Robust ePortfolio Implementation Framework 

This study contributes insight into faculty and 
administrator perspectives regarding a university-wide 
ePortfolio adoption process, as well as a framework for 
supporting it. As noted, faculty perspectives on 


instructional technology adoption, including ePortfolio 
adoption, have not been well documented in the 
literature (Ruiz et al., 2009; Wang & Turner, 2007), and 
yet they are key stakeholders in the process. Findings 
from this project shed light on what faculty and 
administrators value in an ePortfolio implementation 
process including a user-friendly infrastructure, a clear 
and communicated purpose, support and training, and 
rewards for use and time commitment. By creating a 
framework for implementation that acknowledges 
faculty priorities and engages them early and 
systemically in the process can heighten instructional 
technology adoption and sustainment success. In a 
study of Florida State University’s (FSU) ePortfolio 
adoption initiative, Lumsden (2007) identifies 
“feedback and buy-in from key stakeholders (students, 
staff, faculty, and employers)” (44) as a critical success 
factor. Further, components of the framework are 
consistent with several other named success factors of 
the FSU program, including university-wide leadership 
and vision and ePortfolio evaluation. At FSU, 
stakeholders established the goals and created the 
prototype for the ePortfolio system through an iterative, 
design-evaluation-revision process prior to a team of 
information technology professionals beginning work 
on the infrastructure, thus resulting in a more usable 
system (Reardon, Lumsden, and Meyer, 2005), a 
prominent concern of participants in this study. 

A study on instructional technology adoption by 
Lei and Morrow (2010) provides further support for an 
implementation framework that puts faculty at the 
center of a process early on as collaborative decision 
makers and enactors who are well-supported by strong 
leadership, sufficient and timely resources, a means for 
critical feedback, and incentives. The framework makes 
the innovation process transparent to all stakeholders 
and guides open communication and decision-making 
across the organization. Educators remain connected to 
the initiative and are better prepared to call on the most 
important innovation adoption support elements at 
critical and appropriate times. Using a framework that 
supports an open implementation process, educators are 
less distracted by unexpected innovation stumbling 
blocks and better able to focus on ePortfolios as a 
means for student learning, assessment, and 
professional development. 

In the past two decades, educational researchers 
continue to demonstrate that instructional technologies 
have not been well integrated into teaching and learning 
practices (Cuban, 2001; Tyack & Cuban 1995; Zhao, 
Pugh, Sheldon, & Byers, 2002). Significant costs 
associated with wide-scale integration efforts, such as 
organization-wide ePortfolio initiatives, certainly 
warrant concern if investments are not yielding positive 
and sustainable results. Recall that 42% of respondents 
in this study chose to abandon ePortfolio use. Rogers 
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(2003) and other diffusion of innovation experts 
(Watson, Watson, & Reigeluth, 2008) have well 
documented the complex nature of innovation adoption 
and the supports necessary to undergird this type of 
change. The framework presented here accounts for the 
necessary knowledge and skill development of those 
adopting an instructional technology but also provides 
for other vital supports to an innovation implementation 
process, as well as a way to assess current performance 
on these supports in order to move forward in an 
informed manner. Use of such a framework may 
improve our track record with instructional technology 
integration. However, while validated by DOI experts, 
the framework now needs to be tested in the field in a 
variety of contexts. 

Design and Development Research in Practice 

As the popularity of design and development 
research continues to grow, this research project can 
serve as a model for those who are considering or 
currently using this methodology. While there is 
significant guidance regarding design and development 
research (see, for example, Ellis & Levy, 2010; Richey 
& Klein, 2005; Richey & Klein, 2007), the body of 
research using this methodology remains relatively 
small (Richey & Klein, 2014). This study adds to the 
empirical body of knowledge on design and 
development research, providing another example of 
what this methodology can look like in practice. Two 
important lessons were learned by the researchers in 
using this approach: lessons that may help other 
researchers. First, it took more time than anticipated to 
secure expert reviewers and collect their feedback on 
the framework, a critical component of this 
methodology. In addition to gaining the commitment of 
expert reviewers earlier, using a scheduled interview, 
rather than an online survey, to obtain feedback for the 
framework revision phase might be more efficient. 
Second, developing a framework and the supporting 
instructions for use that was mature enough to be 
understood and evaluated by experts was a more 
challenging task than anticipated. Getting feedback on 
early prototypes of the framework from intended users, 
akin to a rapid prototyping approach (Tripp & 
Bichelmeyer, 1990), may expedite the development 
process and lend greater validity to the framework prior 
to releasing it for expert review. The second author has 
inserted this rapid prototyping approach into the 
development phase of a design and development study 
to be carried out over the coming year. 

Two study limitations should be noted. First, the 
setting of the study, a large higher education institution 
with certain ePortfolio resources in place, could be 
viewed as a study limitation. While the researchers 
aimed for a framework that could be adaptable to any 


higher education setting, survey and interview findings 
may have been different if this study had been 
conducted in a different setting. For example, a smaller 
institution with different ePortfolio technologies at hand 
may have yielded different priorities, potentially 
impacting the framework. A second limitation relates to 
when the study was conducted in relation to the 
institution’s adoption lifecycle. The diffusion literature 
points to an s-curve rate of innovation adoption in 
which early on adoption rates are low but increase 
dramatically in later stages and then taper off as time 
passes often due to fewer adopters and even abandoners 
(Rogers, 2003). This study was conducted fairly late 
into the adoption lifecycle, over ten years into the 
university’s ePortfolio initiative, a factor that could 
have impacted participant perceptions. It is hoped that 
the grounding of the study, including instrument 
development, in the diffusion of innovation literature 
provided a useful counterweight to these concerns. 
However, the study of the framework in other higher 
education contexts and at earlier stages of the adoption 
process is needed to shed light on these issues. As a 
first step, the first author is now using the framework 
created in this study to collaborate with faculty on the 
implementation of ePortfolio as an innovation at a mid¬ 
sized university in the United States. 

The field of instructional design and technology 
demands that researchers and practitioners not only 
create new knowledge, but also research and improve 
upon current practices. The use of design and 
development research can assist professionals in the 
study, improvement, and validation of instructional 
design tools and practices (Klein, 2013), serving as a 
useful approach to forming important connections 
between instructional design theory and instructional 
design practice. 
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Appendix 


A Framework to Support Electronic Portfolio Implementation in Higher Education Contexts 


Introduction to the Framework 

Based on survey and interview data from faculty and administrators who have implemented electronic 
portfolios (ePortfolios) at a large research university in the EFnited States and improved upon by suggestions from 
three diffusion of innovation (DOI) expert reviewers, the following framework for implementing ePortfolios was 
created. The framework is meant to support those implementing, or attempting to implement, ePortfolios in a higher 
education context by guiding them through key attributes of systemic innovation in a practical and applied manner. 


i 

/ 


Awareness 

Professional knowledge 
of the pedagogical 
benefits of electronic 
portfolios 




Evaluation 

The data-based 
examination of 
electronic portfolio 
use for improvement 
to future iterations 


Leadership 

The necessary 
leadership support 


m place to sustain 


use of electronic 


portfolios 



Motivation 

Identification and/or 
presence of intrinsic 
and extrinsic incentives 
for electronic portfolio 


Commitment 

The decision, as a result 
of value recognition, to 
implement electronic 
portfolios 


> 

► 


Figure 2. ePortfolio Implementation Framework components 

As illustrated in Figure 2, the framework is divided into six components that are vital to the successful 
implementation of ePortfolios by faculty over time. These components (Awareness, Motivation, Commitment, 
Resources, Leadership, and Evaluation) reflect important DOI elements put forth by Everett M. Rogers (2003) and 
Donald P. Ely (1990), prominent scholars in systemic change. Awareness is defined as professional knowledge of 
the pedagogical benefits of ePortfolios and corresponds with Roger’s element of knowledge as well as Ely’s 
condition of dissatisfaction with the status quo. Motivation is defined as the identification and/or presence of 
intrinsic and/or extrinsic incentives for using ePortfolios and corresponds with Roger’s element of persuasion as 
well as Ely’s conditions of dissatisfaction with the status quo and rewards or incentives. Commitment is defined as 
the decision, as a result of value recognition, to implement ePortfolios and corresponds with Roger’s element of 
decision as well as Ely’s conditions of participation and commitment. Resources is defined as identified resources to 
assist in ePortfolio implementation and corresponds with Roger’s element of implementation as well as Ely’s 
conditions of sufficient knowledge and skills, availability of time, and availability of resources. Leadership is 
defined as the necessary leadership supports in place to sustain use of ePortfolios and corresponds with Roger’s 
element of implementation as well as Ely’s conditions of leadership. Evaluation is defined as the data-based 
examination of ePortfolio use to inform improvements to future iterations and corresponds with Roger’s element of 
confirmation. 

The framework was built to assist those in a higher education context who are considering implementing 
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portfolios or already in the process of implementation to assess a workgroup’s current status in the implementation 
process, as well as important next steps. The framework is modular in that components can be considered in any 
order as needed. In addition to defining each component, selected strategies to act on each component, as well as 
key stakeholders who can influence progress on that component are provided (See Figure 3). In column four of each 
framework component, you may notice a scale for rating the current implementation status of the component. This 
scale is provided for users to assess performance on each component and identify next steps important to 
implementation. The intent of the 3-point rating scale is for the workgroup (e.g. organization, department, or 
program level) to take the pulse of the group’s current implementation status. A rating of one would identify a 
component as a priority in planning efforts, whereas a three would indicate the component is of low priority. 
Through such a quick check, action planning can then be based in top priorities. 


Component 


Selected Strategies to 
Support Component 


Key Stakeholder Rating of Current 

Involvement Implementation Status 


Next Steps for 
Implementation Efforts 




Awareness 




Websites 

Newsletters 

Articles 

Presentations 

Professional 

development 


Professional 
knowledge of the 
pedagogical benefits 
of electronic 
portfolios 


Identified high- 

1. Faculty are unaware 

Rating of 1 or 2 

level opinion 

of the pedagogical 

• Identify multiple avenues for electronic 

leaders including 

value of electronic 

portfolio awareness building. 

but not limited to: 

portfolios. 

• Plan a 3-6 month awareness building 

o Academic 


campaign. 

leaders on 

2. Faculty are 

• Reassess awareness status after one to 

campus (e.g. 

somewhat aware of 

two academic years. 

provost, teaching 

the pedagogical 

and learning 

value of electronic 


directors) 
o Leading 

portfolios. 


electronic 

1 3. Faculty are very 


portfolio 

aware of the 


scholars and 

pedagogical value of 


practitioners 

electronic portfolios. 


o Local faculty 


1 Rating of 3 

innovators 


• Reassess awareness status at next formal, 


systemic evaluation of electronic 
portfolio implementation. (A systemic 
evaluation is recommended every three to 
five years.) 
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Component 


Selected Strategies to 
Support Component 


Key Stakeholder Rating of Current 

Involvement Implementation Status 


Next Steps for 
Implementation Efforts 




Motivation 


Identification and/or 
presence of intrinsic 
and extrinsic 
incentives for 
electronic portfolio 


• Learner incentives 

o Hands-on, applied 
projects 
o Alternative 
assessment 
opportunities 
o Showcases and/or 
competitions 
o Job seeking 
resource 

• Faculty incentives 
o Departmental 

awards 

o Accreditation 
fulfillment 
o Teaching release 
time for 
development 
o Grants and 
monetary 
incentives 

• Communication 
channels (awareness 
campaign, 
professional 
development 
sessions) 


Provost and/or 1. 

academic unit 
decision makers 
Electronic portfolio 
advocates 
Faculty innovators 
Student innovators 2. 


Faculty are unaware 
of the intrinsic and 
extrinsic incentives 
for using electronic 
portfolios. 

Faculty are 
somewhat aware of 
the intrinsic and 
extrinsic incentives 
for using electronic 
portfolios. 


Rating of 1 or 2 

• Identify appropriate and realistic 
complement of learner and faculty 
incentives. 

• Use communication channels to convey 
incentives. 

• Showcase examples of student electronic 
portfolios that are relatable and convey 
incentives. 

• Reassess motivation status after one to 
two academic years. 


Faculty are very 

aware of the intrinsic 

and extrinsic Rating of 3 

incentives for using • Reassess motivation status at next formal, 
electronic portfolios. systemic evaluation of electronic 

portfolio implementation. (A systemic 
evaluation is recommended every three to 
five years.) 


Component 


Selected Strategies to 
Support Component 


Key Stakeholder Rating of Current 

Involvement Implementation Status 


Next Steps for 
Implementation Efforts 




Commitment 


The decision, as a 
result of value 
recognition, to 
implement electronic 
portfolios 


Faculty 

acknowledgements 
and rewards for initial 
investments of time 
and effort 

Exemplar electronic 
portfolios as models 
Dedicated ePortfolio 
staff/office 
Inclusion in strategic 
plan 


Provost and/or 1. 

academic unit 
decision makers 
Electronic 
portfolio 

professional staff 2. 

Faculty leadership 
team 


3. 


Faculty are not 
committed to the use 
of electronic 
portfolios. 

Faculty are 
somewhat 

committed to the use 
of electronic 
portfolios. 

Faculty are very 
committed to the use 
of electronic 
portfolios. 


Rating of 1 or 2 

• Identify a faculty leader by department or 
program, to establish and direct a regular 
schedule of work meetings regarding 
electronic portfolio implementation. 

• Reward faculty for initial time and effort. 

• After defining the direction of the 
initiative, involve technology service 
providers and ePortfolio staff in 
conversation with the faculty team to 
ensure potential: 

o technology solutions are feasible, 
o ePortfolio platforms are feasible and 
will support the features of desired 
models. 

• Reassess commitment status after one to 
two academic years. 


Rating of 3 

• Reassess commitment status at next 
formal, systemic evaluation of electronic 
portfolio implementation. (A systemic 
evaluation is recommended every three to 
five years.) 
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Component 


Selected Strategies to 
Support Component 

Key Stakeholder 
Involvement 

Rating of Current 
Implementation Status 

Next Steps for 
Implementation Efforts 

• Targeted in-house 

• In-house 

1. Faculty are unaware 

Rating of 1 or 2 

professional 

professional 

of the resources 

• Use communication channels (awareness 

development 

development 

available to assist in 

campaign, professional development 

activities 

service providers 

their implementation 

sessions) to convey available resources. 

• Dedicated 

(ePortfolio, 

of electronic 

• Ensure that available resources (including 

Technology support 
• Dedicated Pedagogy 

technologies, 

pedagogy) 

portfolios. 

incentives) are accessible to faculty in 
light of work habits and environments. 

support 

• Faculty innovators 

2. Faculty are 

• Check for other reasons for non-use of 

• Professional 

• Graduate assistants 

somewhat aware of 

resources. 

ePortfolio 

• External electronic 

the resources 

• Reassess resources status after one to two 

organizations 
(AAEEBL, ePIC, 
EPAC) and 
associated resources 

portfolio 
professional 
organizations, 
conferences, 
and/or journals 

available to assist in 
their implementation 
of electronic 
portfolios. 

3. Faculty are very 
aware of the 
resources available 
to assist in their 
implementation of 
electronic portfolios. 

academic years. 

Rating of 3 

• Reassess resources status at next formal, 
systemic evaluation of electronic 
portfolio implementation. (A systemic 
evaluation is recommended every three to 
five years.) 



Identified resources 
to assist in electronic 
portfolio 
implementation 


Component 


Selected Strategies to 
Support Component 


Key Stakeholder Rating of Current 

Involvement Implementation Status 


Next Steps for 
Implementation Efforts 




Leadership 


The necessary 
leadership support in 
place to sustain use 
of electronic 
portfolios 


Ongoing recognition 
by important leaders 
through preferred 
communication 
channels (websites, 
newsletters, 
showcases, and 
presentations) 
Inclusion in strategic 
plan and other policy 
documents 


Electronic 1. 

portfolio 

professional staff 
Contact staff for 
technology and 
pedagogy supports 
Provost and/or 2. 

academic unit 
decision makers 
Faculty champions 

3. 


Faculty do not have 
the leadership 
support needed to 
sustain their use of 
electronic portfolios. 

Faculty somewhat 
have the leadership 
support needed to 
sustain their use of 
electronic portfolios. 


Rating of 1 or 2 

• Align electronic portfolio initiative with 
strategic plan goals. 

• Recognize faculty and student electronic 
portfolio work. 

• Develop new initiatives for enhancing 
and highlighting ongoing ePortfolio 
work. 

• Reassess leadership status after one to 
two academic years. 


Faculty have the 
leadership support 
needed to sustain 
their use of 
electronic portfolios. 


Rating of 3 

• Reassess leadership status at next formal, 
systemic evaluation of electronic 
portfolio implementation. (A systemic 
evaluation is recommended every three to 
five years.) 
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Component 


Selected Strategies to 
Support Component 


Key Stakeholder 
Involvement 


Rating of Current 
Implementation Status 


Next Steps for 
Implementation Efforts 



Gather student, 
faculty, and/or 
potential employer 
feedback through 
survey, interview, or 
other data collection 
options. 


Electronic 

portfolio 

professional staff 

In-house 

evaluation 

resources service 

provider 

Faculty 


The data-based 
examination of 
electronic portfolio 
use for 

improvements to 
future iterations 


1. Faculty are unaware 
of the evaluation 
activities and 
outcomes related to 
the implementation 
of electronic 
portfolios. 

2. Faculty are 
somewhat aware of 
the evaluation 
activities and 
outcomes related to 
the implementation 
of electronic 
portfolios. 

3. Faculty are very 
aware of the 
evaluation activities 
and outcomes 
related to the 
implementation of 
electronic portfolios. 


Rating of 1 or 2 

• Electronic portfolio staff and evaluation 
staff plan for and implement an electronic 
portfolio evaluation 

• Use evaluation findings to improve the 
next phase of ePortfolio implementation 

• Reassess evaluation status after one to 
two academic years. 


Rating of 3 

• Reassess evaluation status at next formal, 
systemic evaluation of electronic 
portfolio implementation. (A systemic 
evaluation is recommended every three to 
five years.) 


Figure 3. Component details of ePortfolio Implementation Framework 
The action planning worksheet in Figure 4 can be used to identify next steps in the implementation process. Project 
management of these steps can then begin through the identification of key stakeholders and target completion dates. 


Component 


Awareness 

Motivation 1 

Commitment 


Current 

Implementation 

Rating 

Next Steps 

Key Players to Involve 

Target Completion 
Date(s) 

Additional Notes 

□ 1 

□ 2 

□ 3 





□ 1 

□2 

□ 3 





□ 1 

□2 

□3 
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Component 

Current 

Implementation 

Rating 

Next Steps 

Key Players to Involve 

Target Completion 
Date(s) 

Additional Notes 

Resources 

□ 1 

□2 

□ 3 





Leadership 

Evaluation 

□ 1 

□ 2 

□ 3 





□ 1 

□2 

□ 3 






Figure 4. Action Planning Worksheet for ePortfolio Implementation 










